Electronic device including camera and method for generating video recording of a moving object

ABSTRACT

An electronic device includes a display, a memory and a processor. The memory includes instructions for causing the processor to, when generating a video, receive a video signal including an object to obtain a plurality of frame images having a first size, estimate a movement trajectory of the object included in the plurality of frame images, stitch together frame images in which overlapping portions of the video signal are arranged to overlap according to the movement trajectory of the object to generate a stitched image having a second size larger than the first size; and store the generated stitched image and the plurality of frame images as a video file.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/KR2023/000162 designating the United States, filed on Jan. 4, 2023,in the Korean Intellectual Property Receiving Office and claimingpriority to Korean Patent Application 10-2022-0002217, filed on Jan. 6,2022, at the Korean Intellectual Property Office, the disclosures ofwhich are incorporated by reference herein in their entireties.

TECHNICAL FIELD

Various embodiments relate to an electronic device including a cameraand a method for generating a video recording of a moving object.

BACKGROUND ART

An electronic device (e.g., a smartphone) provides a function ofrecording images by using at least one camera (e.g., a camera module andan image sensor). Recently, as the number of users who record imageswith a camera of an electronic device increases, various cameraperformances are also improved.

For example, the electronic device supports a panorama function forgenerating one wide-range image (e.g., a panoramic image) by combining aplurality of recorded frame images (or still images) in addition to thefunction to record with various angles of view (or magnification).

However, in such a panoramic recording technique, because the recordingis continuously performed while the recording direction is changed atvarious angles but still images are connected as one image, it isdifficult to record the moving situation of the subject.

Alternatively, the technique of saving and playing the subject with amoving effect (e.g., motion panorama function) using still images is tosave the recorded images to be played back in the form of a video, sothe technique may be a different recording technique from videorecording in which audio is recorded.

DISCLOSURE Technical Problem

Users use a video recording function to record a moving subject.However, when recording an image using an electronic device, a cameraangle of view (in other words, a field of view of a scene captured bythe camera) has a limit, so there is a limit in recording a scenedesired by a user. For example, the electronic device may supportmagnifications such as ×1, ×2, ×10, ×50, and ×100 when taking a pictureor still image, while only supporting a magnification according to asupportable resolution when taking a video.

For example, when an image of a distant moving subject is desired to berecorded, the user may record the subject by adjusting a cameramagnification (e.g., zoom expansion) according to the subject. In thiscase, the subject may be enlarged and recorded, but it may be difficultto record the moving trajectory of the subject as a video because theentire background is excluded from the angle of view of the camera.

On the other hand, when a distant subject is recorded without increasingthe magnification of the camera, image quality (or resolution) may bedeteriorated, so it may be difficult to accurately recognize thesubject. In addition, while the size of the subject is large even whenrecording a close subject, it may be difficult to record a scenerepresenting the movement trajectory of the subject because it isdifficult to record the background due to a narrow angle of view.

Various embodiments may provide a method for generating a video as ascene having a larger range (or a wider range) than a limited cameraangle of view while recording the scene indicating the moving trajectoryof the subject in a more stable state when recording the moving subject.

Technical Solution

In accordance with an aspect of the disclosure, an electronic device isprovided. An electronic device comprises a display, a memory and aprocessor, wherein the memory comprises instructions for causing theprocessor to, when generating a video, receive a video signal includingan object to obtain a plurality of frame images having a first size,estimate the movement trajectory of the object included in the pluralityof frame images, stitch the frame images in which overlapping portionsof the video signal are arranged to overlap according to the movementtrajectory of the object to generate a stitched image having a secondsize larger than the first size; and store the generated stitched imageand the plurality of frame images as a video file.

In accordance with another aspect of the disclosure method of generatinga video by recording a moving object of an electronic device, the methodcomprising receiving a video signal including an object to obtain aplurality of frame images having a first size, when recording a video,estimating the movement trajectory of the object included in theplurality of frame images, generating a stitched image having a secondsize larger than the first size by stitching the frame images in whichoverlapping portions of the video signal are arranged to overlapaccording to the movement trajectory of the object; and storing thegenerated stitched image and the plurality of frame images as a videofile, wherein in the storing, the plurality of frame images are storedto be played based on the generated stitched image.

Advantageous Effects

An electronic device according to various embodiments can store imagesand generate a video having a wide range of scenes so as to effectivelyshow a moving trajectory of a moving subject in recording the movingsubject.

The electronic device according to various embodiments can generate avideo with a stabilized trajectory by removing a noise component of amoving subject, and may record the image so that the image is smoothlyconnected.

The electronic device according to various embodiments can beeffectively applied to a portable electronic device by processing imagesin real-time frame by frame, and adjusting and storing the size of theimage data, even when generating a video having an angle of view largerthan the size of the limited angle of view.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment according to an embodiment of the disclosure.

FIG. 2 is a block diagram illustrating a camera module according tovarious embodiments.

FIG. 3 shows a video generation method of recording a moving object ofan electronic device according to various embodiments.

FIG. 4 shows an example of frame images and stitched images according toan embodiment.

FIG. 5 shows a video generation method of capturing a moving object ofan electronic device according to various embodiments.

FIG. 6A shows motions of an electronic device and a moving object.

FIG. 6B shows an example of smoothing a motion trajectory of a movingobject.

FIG. 6C shows an example of stitched image synthesis.

FIG. 7 shows frame images collected when an image is recorded by anelectronic device according to an exemplary embodiment.

FIG. 8 is a stitched image generated when an image is recorded by theelectronic device according to an exemplary embodiment.

MODE FOR INVENTION

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smartphone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment according to an embodiment of the disclosure.

Referring to FIG. 1 , the electronic device 101 in the networkenvironment 100 may communicate with an electronic device 102 via afirst network 198 (e.g., a short-range wireless communication network),or at least one of an electronic device 104 or a server 108 via a secondnetwork 199 (e.g., a long-range wireless communication network).According to an embodiment, the electronic device 101 may communicatewith the electronic device 104 via the server 108. According to anembodiment, the electronic device 101 may include a processor 120,memory 130, an input module 150, a sound output module 155, a displaymodule 160, an audio module 170, a sensor module 176, an interface 177,a connecting terminal 178, a haptic module 179, a camera module 180, apower management module 188, a battery 189, a communication module 190,a subscriber identification module (SIM) 196, or an antenna module 197.In some embodiments, at least one of the components (e.g., theconnecting terminal 178) may be omitted from the electronic device 101,or one or more other components may be added in the electronic device101. In some embodiments, some of the components (e.g., the sensormodule 176, the camera module 180, or the antenna module 197) may beimplemented as a single component (e.g., the display module 160).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device. 101 coupled with theprocessor 120, and may perform various data processing or computation.According to one embodiment, as at least part of the data processing orcomputation, the processor 120 may store a command or data received fromanother component (e.g., the sensor module 176 or the communicationmodule 190) in volatile memory 132, process the command or the datastored in the volatile memory 132, and store resulting data innon-volatile memory 134. According to an embodiment, the processor 120may include a main processor 121 (e.g., a central processing unit (CPU)or an application processor (AP)), or an auxiliary processor 123 (e.g.,a graphics processing unit (GPU), a neural processing unit (NPU), animage signal processor (ISP), a sensor hub processor, or a communicationprocessor (CP)) that is operable independently from, or in conjunctionwith, the main processor 121. For example, when the electronic device101 includes the main processor 121 and the auxiliary processor 123, theauxiliary processor 123 may be adapted to consume less power than themain processor 121, or to be specific to a specified function. Theauxiliary processor 123 may be implemented as separate from, or as partof the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display module 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an image signal processoror a communication processor) may be implemented as part of anothercomponent (e.g., the camera module 180 or the communication module 190)functionally related to the auxiliary processor 123. According to anembodiment, the auxiliary processor 123 (e.g., the neural processingunit) may include a hardware structure specified for artificialintelligence model processing. An artificial intelligence model may begenerated by machine learning. Such learning may be performed, e.g., bythe electronic device 101 where the artificial intelligence is performedor via a separate server (e.g., the server 108). Learning algorithms mayinclude, but are not limited to, e.g., supervised learning, unsupervisedlearning, semi-supervised learning, or reinforcement learning. Theartificial intelligence model may include a plurality of artificialneural network layers. The artificial neural network may be a deepneural network (DNN), a convolutional neural network (CNN), a recurrentneural network (RNN), a restricted boltzmann machine (RBM), a deepbelief network (DBN), a bidirectional recurrent deep neural network(BRDNN), deep Q-network or a combination of two or more thereof but isnot limited thereto. The artificial intelligence model may, additionallyor alternatively, include a software structure other than the hardwarestructure.

The memory 130 may store various data used by at least one component(e.g., the processor 120 or the sensor module 176) of the electronicdevice 101. The various data may include, for example, software (e.g.,the program 140) and input data or output data for a command relatedthereto. The memory 130 may include the volatile memory 132 or thenon-volatile memory 134.

The program 140 may be stored in the memory 130 as software, and mayinclude, for example, an operating system (OS) 142, middleware 144, oran application 146.

The input module 150 may receive a command or data to be used by anothercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputmodule 150 may include, for example, a microphone, a mouse, a keyboard,a key (e.g., a button), or a digital pen (e.g., a stylus pen.

The sound output module 155 may output sound signals to the outside ofthe electronic device 101. The sound output module 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record. The receiver maybe used for receiving incoming calls. According to an embodiment, thereceiver may be implemented as separate from, or as part of the speaker.

The display module 160 may visually provide information to the outside(e.g., a user) of the electronic device 101, The display module 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaymodule 160 may include a touch sensor adapted to detect a touch, or apressure sensor adapted to measure the intensity of force incurred bythe touch.

The audio module 170 may convert a sound into an electrical signal andvice versa, According to an embodiment, the audio module 170 may obtainthe sound via the input module 150, or output the sound via the soundoutput module 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HMV, a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to one embodiment, the power managementmodule 188 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device. 104, or the server 108)and performing communication via the established communication channel.The communication module 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication module 190 may include a wireless communication(nodule 192 (e.g., a cellular communication module, a short-rangewireless communication module, or a global navigation satellite system(GNSS) communication module) or a wired communication module 194 (e.g.,a local area network (LAN) communication module or a power linecommunication (PLC) module). A corresponding one of these communicationmodules may communicate with the external electronic device via thefirst network 198 (e.g., a short-range communication network, such asBluetooth™, wireless-fidelity (Wi-Fi) direct, or infrared dataassociation (IrDA)) or the second network 199 (e.g., a long-rangecommunication network, such as a legacy cellular network, a 50 network,a next-generation communication network, the Internet, or a computernetwork (e.g., LAN or wide area network (WAN)). These various types ofcommunication modules may be implemented as a single component (e.g., asingle chip), or may be implemented as multi components (e.g., multichips) separate from each other. The wireless communication module 192may identify and authenticate the electronic device 101 in acommunication network, such as the first network 198 or the secondnetwork 199, using subscriber information (e.g., international mobilesubscriber identity (IMSI)) stored in the subscriber identificationmodule 196.

The wireless communication module 192 may support a 50 network, after a4G network, and next-generation communication technology, e.g., newradio (NR) access technology. The NR access technology may supportenhanced mobile broadband (eMBB), massive machine type communications(mMTC), or ultra-reliable and low-latency communications (URLLC). Thewireless communication module 192 may support a high-frequency band(e.g., the mmWave band) to achieve, e.g., a high data transmission rate.The wireless communication module 19:2 may support various technologiesfor securing performance on a high-frequency band, such as, e.g.,beamforming, massive multiple-input and multiple-output (massive MIMO),full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, orlarge scale antenna. The wireless communication module 192 may supportvarious requirements specified in the electronic device 101, an externalelectronic device (e.g., the electronic device 104), or a network system(e.g., the second network 199). According to an embodiment, the wirelesscommunication module 192 may support a peak data rate (e.g., 20 Gbps ormore) for implementing eMBB, loss coverage (e.g., 164 dB or less) forimplementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each ofdownlink (DL) and uplink (UL), or a round trip of 1 ms or less) forimplementing URLLC

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include an antenna including a radiating element composed of aconductive material or a conductive pattern formed in or on a substrate(e.g., a printed circuit board (PCB)). According to an embodiment, theantenna module 197 may include a plurality of antennas (e.g., arrayantennas). In such a case, at least one antenna appropriate for acommunication scheme used in the communication network, such as thefirst network 198 or the second network 199, may be selected, forexample, by the communication module 190 (e.g., the wirelesscommunication module 192) from the plurality of antennas. The signal orthe power may then be transmitted or received between the communicationmodule 190 and the external electronic device via the selected at leastone antenna. According to an embodiment, another component (e.g., aradio frequency integrated circuit (RFIC)) other than the radiatingelement may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form ammWave antenna module. According to an embodiment, the mmWave antennamodule may include a printed circuit board, a RFIC disposed on a firstsurface (e.g., the bottom surface) of the printed circuit board, oradjacent to the first surface and capable of supporting a designatedhigh-frequency band (e.g., the mmWave band), and a plurality of antennas(e.g., array antennas) disposed on a second surface (e.g., the top or aside surface) of the printed circuit board, or adjacent to the secondsurface and capable of transmitting or receiving signals of thedesignated high-frequency band.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the electronic devices 10:2 or 104 may be a device of a same type as,or a different type, from the electronic device 101. According to anembodiment, all or some of operations to be executed at the electronicdevice 101 may be executed at one or more of the external electronicdevices 102, 104, or 108. For example, if the electronic device 101should perform a function or a service automatically, or in response toa request from a user or another device, the electronic device 101,instead of, or in addition to, executing the function or the service,may request the one or more external electronic devices to perform atleast part of the function or the service. The one or more externalelectronic devices receiving the request may perform the at least partof the function or the service requested, or an additional function oran additional service related to the request, and transfer an outcome ofthe performing to the electronic device 101. The electronic device 101may provide the outcome, with or without further processing of theoutcome, as at least part of a reply to the request. To that end, acloud computing, distributed computing, mobile edge computing (MEC), orclient-server computing technology may be used, for example. Theelectronic device 101 may provide ultra low-latency services using,e.g., distributed computing or mobile edge computing. In anotherembodiment, the external electronic device 104 may include anInternet-of-things (IoT) device. The server 108 may be an intelligentserver using machine learning and/or a neural network. According to anembodiment, the external electronic device 104 or the server 108 may beincluded in the second network 199. The electronic device 101 may beapplied to intelligent services (e.g., smart home, smart city, smartcar, or healthcare) based on 5G communication technology or IoT-relatedtechnology.

FIG. 2 is a block diagram 200 illustrating the camera module 180according to various embodiments.

Referring to FIG. 2 , the camera module 180 may include a lens assembly210, a flash 220, an image sensor 230, an image stabilizer 240, memory250 (e.g., buffer memory), or an image signal processor 260.

The lens assembly 210 may collect light emitted or reflected from anobject whose image is to be taken. The lens assembly 210 may include oneor more lenses. According to an embodiment, the camera module 180 mayinclude a plurality of lens assemblies 210. In such a case, the cameramodule 180 may form, for example, a dual camera, a 360-degree camera, ora spherical camera. Some of the plurality of lens assemblies 210 mayhave the same lens attribute (e.g., view angle, focal length,auto-focusing, f number, or optical zoom), or at least one lens assemblymay have one or more lens attributes different from those of anotherlens assembly. The lens assembly 210 may include, for example, awide-angle lens or a telephoto lens.

The flash 220 may emit light that is used to reinforce light reflectedfrom an object. According to an embodiment, the flash 220 may includeone or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB)LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or axenon lamp. The image sensor 230 may obtain an image corresponding to anobject by converting light emitted or reflected from the object andtransmitted via the lens assembly 210 into an electrical signal.According to an embodiment, the image sensor 230 may include oneselected from image sensors having different attributes, such as a RGBsensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, aplurality of image sensors having the same attribute, or a plurality ofimage sensors having different attributes. Each image sensor included inthe image sensor 230 may be implemented using, for example, a chargedcoupled device (CCD) sensor or a complementary metal oxide semiconductor(CMOS) sensor

The image stabilizer 240 may move the image sensor 230 or at least onelens included in the lens assembly 210 in a particular direction, orcontrol an operational attribute (e.g., adjust the read-out timing) ofthe image sensor 230 in response to the movement of the camera module180 or the electronic device 101 including the camera module 180. Thisallows compensating for at least part of a negative effect (e.g., imageblurring) by the movement on an image being captured.

According to an embodiment, the image stabilizer 240 may sense such amovement by the camera module 180 or the electronic device 101 using agyro sensor (not shown) or an acceleration sensor (not shown) disposedinside or outside the camera module 180. According to an embodiment, theimage stabilizer 240 may be implemented, for example, as an opticalimage stabilizer.

The memory 250 may store, at least temporarily, at least part of animage obtained via the image sensor 230 for a subsequent imageprocessing task. For example, if image capturing is delayed due toshutter lag or multiple images are quickly captured, a raw imageobtained (e.g., a Bayer-patterned image, a high-resolution image) may bestored in the memory 250, and its corresponding copy image (e.g., alow-resolution image) may be previewed via the display module 160.Thereafter, if a specified condition is met (e.g., by a user's input orsystem command), at least part of the raw image stored in the memory 250may be obtained and processed, for example, by the image signalprocessor 260. According to an embodiment, the memory 250 may beconfigured as at least part of the memory 130 or as a separate memorythat is operated independently from the memory 130.

The image signal processor 260 may perform one or more image processingwith respect to an image obtained via the image sensor 230 or an imagestored in the memory 250. The one or more image processing may include,for example, depth map generation, three-dimensional (3D) modeling,panorama generation, feature point extraction, image synthesizing, orimage compensation (e.g., noise reduction, resolution adjustment,brightness adjustment, blurring, sharpening, or softening). Additionallyor alternatively, the image signal processor 260 may perform control(e.g., exposure time control or read-out timing control) with respect toat least one (e.g., the image sensor 230) of the components included inthe camera module 180. An image processed by the image signal processor260 may be stored back in the memory 250 for further processing, or maybe provided to an external component (e.g., the memory 130, the displaymodule 160, the electronic device 102, the electronic device 104, or theserver 108) outside the camera module 180.

According to an embodiment, the image signal processor 260 may beconfigured as at least part of the processor 120, or as a separateprocessor that is operated independently from the processor 120. If theimage signal processor 260 is configured as a separate processor fromthe processor 120, at least one image processed by the image signalprocessor 260 may be displayed, by the processor 120, via the displaymodule 160 as it is or after being further processed.

According to an embodiment, the electronic device 101 may include aplurality of camera modules 180 having different attributes orfunctions. In such a case, at least one of the plurality of cameramodules 180 may form, for example, a wide-angle camera and at leastanother of the plurality of camera modules 180 may form a telephotocamera. Similarly, at least one of the plurality of camera modules 180may form, for example, a front camera and at least another of theplurality of camera modules 180 may form a rear camera.

According to various embodiments, the electronic device 101 may supporta function (e.g., panorama video function) of recording a video havingan angle of view that is greater than an angle of view supported by thecamera when recording a video.

According to various embodiments, the electronic device (e.g., anelectronic device 101 of FIG. 1 ) may include a display (e.g., a displaymodule 160 of FIG. 1 ), a memory (e.g., a memory 130 of FIG. 1 ), and aprocessor (e.g., a processor 120 of FIG. 1 ), and the memory 130 mayinclude instructions, when recording a video, for causing the processor120 to receive a video signal including an object to obtain a pluralityof frame images having a first size, estimate the movement trajectory ofthe object included in the plurality of frame images, stitch the frameimages in which overlapping portions of the video signal are arranged tooverlap according to the movement trajectory of the object to generate astitched image having a second size larger than the first size, andstore the generated stitched image and the plurality of frame images asa video file.

According to various embodiments, the memory 130 may further includeinstructions configured to cause the processor 120 to store the frameimages to be played, based on the generated stitched image, and call thestitched image of the second size to the display when playing the videofile, the display data of the stitched image being generated using pixeldata of the frame images.

According to various embodiments, the memory may further includeinstructions configured to cause the processor to identify a distanceaccording to a movement trajectory of the object and predict the size ofthe stitched image when frame images are synthesized according to themovement trajectory, and scale the frame images to fit the size of thestitched image.

According to various embodiments, it is characterized in that thestitched image may be an image obtained by arranging the plurality offrame images in a two-dimensional or three-dimensional space, andstitching overlapping portions of adjacently arranged frame images.

According to various embodiments, it is characterized in that thestitched image may be generated in a size having four vertices where theoutermost boundary lines of frame images in which parts having the sameimage signal among the frame images are arranged in an overlapping formin a two-dimensional image are extended and met.

According to various embodiments, the memory 130 may further includeinstructions configured to cause the processor 120 to extract the motionvector of the object motion according to the position of the object foreach frame image, verify whether the motion vectors are valid, identifyvalidated object motions based on the verified motion vector, andestimate the movement trajectory by connecting the validated objectmotions.

According to various embodiments, the memory 130 may further includeinstructions configured to cause the processor 120 to calculate theamount of change of a motion vector corresponding to the motion of theobject corresponding to the movement of the object, and determinewhether the amount of change is equal to or greater than a set thresholdto verify validity, for each frame image.

According to various embodiments, the memory 130 may further includeinstructions configured to cause the processor 120 to exclude motionvector data of the invalid object motion and generate replacement datato replace the invalid object motion, when it is determined that theobject motion is invalid, as a result of the validation, and estimatethe movement trajectory of the object, based on the data verified as thevalid object motion and the replacement data.

According to various embodiments, the memory 130 may further includeinstructions configured to cause the processor 120 to perform trajectorysmoothing with the object motion in which the movement trajectory issmoothed by removing small shake data of the movement trajectory.

According to various embodiments, the memory 130 may further includeinstructions configured to cause the processor 120 to calculate acorrection value for scaling the size of all frame images according tothe size of the stitched image, and scale the size of all frame imagesby applying the correction value.

FIG. 3 shows a video generation method of recording a moving object ofan electronic device according to various embodiments, and FIG. 4 showsan example of frame images and stitched images according to anembodiment.

Referring to FIGS. 3 and 4 , the processor 120 of the electronic device101 according to an embodiment may activate a camera to start recordinga video in operation 310.

In operation 320, the processor 120 may receive an image signal (e.g.,pixel data) including an object (e.g., a subject (e.g., the subject 410in FIG. 4 )) from at least one camera or an image sensor included in thecamera to obtain a plurality of frame images (hereinafter, may bereferred to as a captured image or a raw image) (e.g., frame images 420of FIG. 4 ) having a first size in response to the start of videorecording.

The first size may be a size corresponding to a frame angle of view (orrecording range) set for image recording. The frame angle of view maycorrespond to the zoom factor information of the camera lens.

The frame images 420 may refer to data before being scaled to a displaysize set in the display (e.g., the display device 160 of FIG. 1 ), andmay be temporarily stored in the memory (e.g., a buffer) in units offrames.

The processor 120 may arrange the frame images for each frame accordingto the received order based on the received image signal. The frameimages 420 may be referred to as frame image 1 (420-1), frame image 2(420-2), etc., up to frame image n (420-n) according to the order oftime. The above-described example is set in the order of increasingnumbers according to the order of time, but is not limited thereto, andmay be set in the reverse order or in any order. The frame images 420stored in units of frames may include time information (e.g., timestamp) for each frame.

In operation 330, the processor 120 may estimate the movement trajectoryof the subject 410 included in the plurality of frame images 420.

According to an embodiment, the processor 120 may estimate the movementtrajectory of the subject by performing an object motion estimationoperation, a validation operation, and a trajectory calculationoperation.

The object motion estimation may be an operation of analyzing the motionof the subject 410 and extracting motion vectors (e.g., direction andmagnitude values) corresponding to the position of the subject 410.

The validation operation may be an operation of determining whether amotion vector estimated by the motion of the subject 410 is valid. Forexample, the processor 120 may calculate the change amount of the pixelvalue corresponding to the movement of the subject 410 for each frameimage, determine whether the change amount is equal to or greater than aset threshold value, and verify whether the change amount is valid. Thatis, when the change amount is equal to or greater than the set thresholdvalue, the change amount is determined to be valid.

The trajectory calculation may be an operation of calculating themovement trajectory of the subject 410 by connecting the motion vectoraccording to the position of the subject 410 in the plurality of frameimages 420 by reflecting the validation result. For example, theprocessor 120 may arrange the frame images 420 in a three-dimensionalspace, and calculate a movement trajectory by connecting the motionvectors of each subject 410 in the frame images 420 arranged in athree-dimensional space.

According to various embodiments, the processor 120 may predict the sizeof an image (e.g., a stitched image) synthesized by stitching the frameimages 420 based on the movement trajectory of the subject 410.

In operation 340, the processor 120 may generate a stitched image 430having a second size having a range larger than the first size bystitching frame images according to the movement trajectory of thesubject.

The processor 120 may generate the stitched image 430 by performing astitching size prediction, a trajectory smoothing operation, and ascaling operation.

The stitching size prediction may be an operation of identifying thedistance according to the movement trajectory of the subject 410included in the frame images 420 collected over time, and predicting thesize of a stitched image when the frame images 420 are stitched andsynthesized according to a movement trajectory.

The trajectory smoothing (or may be referred to as a trajectorystabilization operation) may be an operation of generating a smoothingmovement trajectory by removing small shake (or shake) data in order tosmooth the movement trajectory of the subject 410.

The scaling operation may be an operation of scaling the sizes of allframe images 420 according to the size of the stitched image 430.

In <4001> of FIG. 4 , the frame images 420 may be arranged inoverlapping form with portions having the same image signal (that is,pixel data), and may start at frame image 1(420-1) and end at frameimage (420-n) in time series.

In <4002> of FIG. 4 , a stitched image 430 synthesized by stitchingframe images 420 arranged according to the movement trajectory of thesubject 410 as in FIG. 4 is illustrated. The stitched image 430 isgenerated in the size of the display area displayed on the display, andmay have a width (a) and a height (b).

For example, the stitched image may have a size having four verticesthat extend and meet the outermost boundary lines of frame images inwhich parts having the same image signal (that is, pixel data) among theframe images are arranged in a superimposed two-dimensional image (e.g.,plane).

The stitched image 430 may include a frame area 430 a having pixel databy stitching the frame images 420 (that is, a portion arranged in frameimages) and a blank area where no image signal is collected (or a darkregion) 430 b.

According to an embodiment, the processor 120 may fill the display dataof the blank region 430 b based on the pixel data of the frame region430 a based on a boundary line between the blank region 430 b and maygenerate a stitched image in which the blank area 430 b is filled.

In operation 350, the processor 120 may generate an image file (or apanoramic video) by storing the frame images 420 to be reproduced, basedon the stitched image 430.

According to an embodiment, the processor 120 may generate an image fileby compressing and storing the frame images 420 and the at least onestitched image 430 into one image file.

According to an embodiment, the processor 120 may place the stitchedimage 430 in the first frame (e.g., frame image 1 (420-1)) or the lastframe (e.g., frame image n (420-n)) of the frame images 420, and thenmay store the frame images 420.

According to an embodiment, the processor 120 may generate a pluralityof stitched images, compress and store the plurality of stitched imagesinto a single image file to generate an image file. For example, theprocessor 120 generates a first stitched image with frame images (e.g.,frame images 1 and 2 (420-1 and 420-2)) collected at every set time,generate a second stitched image with the re-collected frame images(e.g., frame images 3 and 4 (420-3 and 420-4)), post-process overlappingportions in a plurality of generated stitched images, and then generateone image file.

According to one embodiment, the processor 120 may store positioninformation of the stitched image 430, a motion vector of each frameimage 420, and a correction value (or scale value) of each frame image420 matching the stitched image 430 as metadata in the image file, inaddition to the frame images 420 and the stitched image 430.

According to an embodiment, the processor 120 may store the stitchedimage 430 to be called on the display, based on the image reproduction,and generate display data of the stitched image 430 using pixel data ofthe frame images 420.

The electronic device 101 may display the stitched image 430 on thedisplay when the image file is reproduced.

FIG. 5 shows a video generation method of capturing a moving object ofan electronic device according to various embodiments, FIG. 6A showsmotions of an electronic device and a moving object, FIG. 6B shows anexample of smoothing a motion trajectory of a moving object, and FIG. 6Cshows an example of stitched image synthesis.

Referring to FIG. 5 , a processor 120 of the electronic device 101according to an embodiment may activate a camera to start recording avideo, in operation 510.

In an embodiment, the electronic device 101 may support a function ofrecording an image having an angle of view larger than that supported bythe camera when recording an image (e.g., a panoramic video function),and when such a function is selected, the process of FIG. 5 may bestarted.

In operation 520, in response to a request to start image recording, theprocessor 120 may receive an image signal (e.g., pixel data) including amoving object (e.g., a subject (e.g., the subject 410 in FIG. 4 )) fromat least one camera or an image sensor included in the camera to obtaina plurality of frame images having a first size (hereinafter, may bereferred to as a captured image or a raw image).

The first size may be a size corresponding to a frame angle of view (orrecording range) set for image capturing. The frame angle of view maycorrespond to the zoom factor information of the camera lens.

The frame images may mean data before being scaled to a display size setin a display (e.g., the display device 160 of FIG. 1 ), and may betemporarily stored in a memory (e.g., a buffer) in units of frames.

The processor 120 may arrange the frame images for each frame accordingto the received order based on the received image signal. The frameimages may be referred to as frame image 1, frame image 2 . . . , etc.,and frame image n in chronological order. The above-described example isset in the order of increasing numbers according to the order of time,but is not limited thereto, and may be set in the reverse order or inany order. The frame images stored in units of frames may include timeinformation (e.g., time stamp) for each frame.

As another example, when the electronic device 101 includes a pluralityof cameras, the processor 120 may obtain a plurality of frame images bycombining image signals obtained from the plurality of cameras.

According to an embodiment, the processor 120 may process the collectedframe images into an image signal (e.g., resizes) to generate a previewimage (hereinafter, may be referred to as a display image) scaled to adisplay size of the display, and display the same on the display.

The user may identify the subject being recorded through the previewimage. When the zoom magnification of the preview image is adjusted, theangle of view of the frame images may be adjusted and the first size maybe changed.

In operation 530, the processor 120 may perform object motion estimation(i.e., a process of calculating a moving distance of a subject in a realarea through a subject spanning a plurality of frame images), based on asubject included in the plurality of frame images.

According to an embodiment, before performing the motion estimationoperation, the processor 120 may identify a moving object (hereinafter,referred to as a subject) according to a user selection or a setcriterion. For example, when the user designates a region of interest(or focusing region) including a subject, the processor 120 may identifythe subject. For another example, the processor 120 may recognize anobject, such as a person, an animal, or an object, and identify it as asubject.

The processor 120 may analyze the movement (or change information ofspecific points) of a subject identified over a plurality of frameimages through an optical flow motion estimation method, and extract thesame as a motion vector (e.g., a direction and a magnitude value). Themotion estimation method may be calculated using a difference in changeof a histogram, a difference in edge change, and/or a difference betweenframes, or may be estimated based on other criteria. Because the motionestimation method is a known technique, a detailed description thereofwill be omitted, and any known or to be developed techniques for motionestimation may be used in addition to the above-described method(s).

In relation to motion estimation, when recording a moving subjectthrough the electronic device (or camera), the electronic device 101 mayalso move along with the moving subject separately from the movement ofthe subject. For example, when it is assumed that the electronic device101 and the subject 610 move as in <6001> of FIG. 6A, the amount ofmovement of the electronic device 101 may be defined as a device motion(or global motion), and the amount of movement of the subject 610 in anactual area may be defined as an object motion.

In this case, because the motion (e.g., device motion) of the electronicdevice 101 during image recording is greater than the object motion ofthe subject 610 moving in the real area when capturing an image(indicated by the size of the arrows illustrated in view <6001>), themotion of the subject 610 shown on the frame images may be divided intoa local motion and a device motion.

For example, as shown in <6002> of FIG. 6A, when the subject 610 movesfrom the first position to the second position between the frame imagesn−1 (620-1 or (N−1)th Frame) and the frame images n (620-2 or (N)thFrame), the electronic device 101 may calculate the local motion and thedevice motion within the camera area. The local motion may refer to themovement of frames due to the difference between frame images within thecamera area, and the device motion may refer to the motion of the ROI(region of interest) including the subject in the camera area. Accordingto an embodiment, the processor 120 may calculate a device motion (e.g.,posture information or direction information), based on informationobtained from a sensor module (e.g., a gyro sensor or an acceleratorsensor) of the electronic device 101. The device motion may be a valuemeasured through a sensor module. The processor 120 may perform thecalculation through image processing.

The processor 120 may calculate (or backtrack) the object motion of themoving subject using difference data (e.g., object motion=devicemotion−local motion) between a device motion representing the motion ofthe electronic device 101 and a local motion of a moving subject. Theobject motion may refer to the amount of movement of the subject 610 inthe real area, and may be a calculated value that the electronic device101 traces back to obtain the movement trajectory of the subject 610 inthe real area.

According to some embodiments, because shaking of the electronic devicemay affect detection of a local motion, the processor 120 may obtain anobject motion by performing compensation processing for the motion ofthe global motion.

In operation 540, the processor 120 may perform validity verification ofthe motion vector for a subject calculated by object motion estimationfor each frame image.

The processor 120 may process or perform an operation for determiningwhether the movement of the subject 410 included in each frame image isvalid.

In an example of the validity verification, the processor 120 maycalculate a change amount of a motion vector corresponding to an objectmotion corresponding to the movement of a subject for each frame image,determine whether the change amount is equal to or greater than a setthreshold value, and verify whether the change amount is valid.

The processor 120 may exclude (or drop) an invalid motion vector andgenerate replacement data (e.g., an alternative motion vector) toreplace the invalid motion vector. For example, the processor 120 maygenerate replacement data, based on a frame image including an invalidmotion vector and frame images having continuity in time series (e.g., amotion vector of a previous frame frame). For another example, theprocessor 120 may generate replacement data using a device motion or anaverage value of motion vectors of frames. As such, in some embodiments,the data representative of the movement trajectory of the subject may bea combination of valid (i.e., verified) object motion data andreplacement data that is generated to replace or substitute for invalidobject motion data.

In operation 550, the processor 120 may calculate or estimate themovement trajectory of the subject.

According to an embodiment, the processor 120 may calculate or estimatethe movement trajectory of the subject based on the valid object motionand replacement data.

In operation 555, the processor 120 may predict the size of an image(e.g., a stitched image) synthesized by stitching the frame images 420based on the movement trajectory of the subject 410.

According to an embodiment, the processor 120 may identify the distanceaccording to the movement trajectory of the subject 410 included in theframe images 420 collected over time, and may process an operation ofpredicting a size of a stitched image when the frame images 420 arestitched and synthesized according to a movement trajectory.

Here, the stitched image may have a size having four vertices where theoutermost boundary lines of frame images in which parts having the sameimage signal (i.e., pixel data) among the frame images are arranged inan overlapping form in a two-dimensional image (e.g., plane) areextended and met. The processor 120 may predict a width (a) and a height(b) of the stitched image (e.g., as shown in FIG. 4 ).

The size prediction of the stitch image may be changed each time thecollected frame images are accumulated.

On the other hand, when the size of the stitched image is graduallyincreased as frame images are collected indefinitely, there may be aproblem in that the size of data is changed to a large capacity whengenerating an image file. By predicting the size of the stitched image,the electronic device may support a function in which the processorflexibly responds so that the data size of a future video image (e.g.,including a second image generated based on the first images) does nothave a capacity greater than or equal to a threshold.

According to an embodiment, when it is predicted that the data size ofthe image file exceeds a value set as a limit as the size of thestitched image increases, the electronic device 101 may display guideinformation for guiding the end of image capturing due to data sizelimitation on a display (e.g., a recording screen).

In operation 560, the processor 120 may determine whether an imagecapturing end request is received.

According to an embodiment, operation 560 may be changed to a positionbetween operation 520 and operation 530, and after operation 520, animage recording end request may be received. In this case, operations530 to 550 may be performed at the end of image recording.

In operation 570, the processor 120 may scale all frame images based onthe size of the stitched image.

According to an embodiment, although not shown in the drawing, beforescaling (or changing) the size of all frame images according to the sizeof the stitched image, trajectory smoothing the moving trajectory of thesubject (or trajectory stabilizing processing) may be performed.

For example, when trajectory smoothing, the processor 120 may removejitter data from the accumulated movement trajectory of the subject,based on a difference between frames and obtain a smoothing motionvector. The trajectory smoothing may use at least one of a video digitalimage stabilizer (VDIS) method, a low pass filter (LPF) method, and anaverage filter method.

For example, FIG. 6B lists frame images 620-1 and 620-n in chronologicalorder, and shows positions and movement trajectories of subjectsincluded in each frame image. View <6003> of FIG. 6B represents amovement trajectory MV1 in which the frame images 620-1 and 620-n,collected during video shooting, are arranged in a three-dimensionalform, and motion vectors of the subject 610 included in the frame imageN (620-n) from the frame image 1 (620-1) are connected. In the movementtrajectory of MV1 view <6003>, it can be seen that the movementtrajectory of the subject is not smooth.

The processor 120 may remove and correct the blur data from the movementtrajectory MV1 that connects the motion vectors of the subject 610 toobtain a movement trajectory MV2 having a smoothing motion vector asshown in view <6004> of FIG. 6B.

The processor 120 may identify a size of a stitch image according to amovement trajectory MV2 having a smoothing motion vector, and scale allframe images according to the stitched image.

According to an embodiment, the processor 120 may calculate a correctionvalue (or scale value) for scaling the size of all frame imagesaccording to the size of the stitched image (e.g., a width (WIDTH) and aheight value), and scale the sizes of all frame images by applying acorrection value.

For example, view <6005> of FIG. 6C represents frame images 620-1 to620-n arranged in a 3D space along a motion vector MV2 having asmoothing motion vector of the subject 610. Screen or image 630 a may bea screen in which overlapping portions of the frame images 620-1 to620-n arranged in a 3D space are overlapped and displayed as one image.

However, the image recorded using the camera of the electronic device101 may be obtained by projecting points on a three-dimensional space toa two-dimensional image (e.g., a plane), and the electronic device 101may generate one stitched image 630 by synthesizing the 2D images, asshown in view <6006> of FIG. 6C.

View <6006> of FIG. 6C represents frame images 620-1 to 620-n in which amotion vector MV3 having a smoothing motion vector of the subject 610 isarranged on a 2D plane rather than a 3D space. When each frame image isarranged on a two-dimensional plane, the x, y, and z axes may becorrected such as position adjustment, distortion, and rotation, and acorrection value may be calculated according to the corrected coordinateinformation. The processor 120 may scale the sizes of all frame imagesbased on the correction values of the respective frame images.

In operation 580, the processor 120 may stitch the frame images arrangedon a two-dimensional plane according to the movement trajectory of thesubject to generate a stitched image of a second size having a rangelarger than the first size.

For example, the stitched image may have a size having four verticesextending and meeting the outermost boundary lines of frame images inwhich parts having the same image signal (i.e., pixel data) among theframe images are arranged in an overlapping form in a two-dimensionalimage (e.g., plane).

The stitched image may include a frame region (i.e., a portion arrangedin frame images) in which the frame images 630 are stitched to havepixel data and a blank region (or dark region) in which an image signalis not collected.

According to an embodiment, the processor 120 may fill the display dataof the blank region based on pixel data having the frame region as aboundary line between the blank region and the frame region 430 a, andmay generate a stitched image in which the blank region is filled.

According to some embodiments, the processor 120 may perform anoperation of performing image compensation (e.g., noise reduction,resolution adjustment, brightness adjustment, blurring, sharpening, orsoftening) on the stitched images.

In operation 590, the processor 120 may generate an image file bystoring frame images to be reproduced based on the stitched image.

According to an embodiment, the processor 120 may generate an image fileby compressing and storing the frame images and the stitched image intoone image file.

According to an embodiment, the processor 120 may store the frame imagesafter positioning the stitched image 430 in the first frame (e.g., frameimage 1) or the last frame (e.g., frame image n) of the frame images.

According to an embodiment, the processor 120 may store positioninformation of the stitched image, a motion vector of each frame image,and a correction value of each frame image matching the stitched imageas metadata in the image file, in addition to the frame images and thestitched image.

According to an embodiment, the processor 120 may store display data tobe displayed on the display, based on the stitched image 430 whenreproducing the image, generate the display data of the stitched imageusing pixel data of the frame images 420, and display the display dataon the display.

FIG. 7 shows frame images collected when an image is recorded by anelectronic device according to an exemplary embodiment, and FIG. 8 is astitched image generated when an image is recorded by the electronicdevice according to an exemplary embodiment.

Referring to FIGS. 7 and 8 , the electronic device 101 according tovarious embodiments may record an image of a moving subject 710according to a user's request.

When image capturing starts, the electronic device 101 may display arecorded image including the subject 710 on the display, based on animage signal displayed from the camera. The recorded image displayed onthe display may be a preview image.

The electronic device may obtain a plurality of frame images 720-1,720-2, . . . , 720-n−1, 720-n, accumulated over time by storing theimage signal received from the camera in frame units when recording animage. When it is assumed that the first received frame image is 720-1,then the last received frame image may be referred to as 720-n. Theframe images may have a first size due to a limitation in the range ofthe recording angle of view.

The electronic device 101 may use frame images of the first size togenerate a stitched image of a second size expressed in a recordingrange (or angle of view) larger than the first size, and store thestitched image as an image file.

Accordingly, even when the electronic device records the image displayedon the display as shown in FIG. 7 , when the recorded image isreproduced, as shown in FIG. 8 , the same image 820 as that recordedwith a recording range larger than the first size may be played. Throughthe image 820 of FIG. 8 , the user may identify the movement trajectoryof the moving subject 810 included in the image on a wider screen.

According to various embodiments, a method of generating a video byrecording a moving object of an electronic device may be characterizedto include receiving a video signal including an object to obtain aplurality of frame images having a first size, when recording a video,estimating the movement trajectory of the object included in theplurality of frame images, generating a stitched image having a secondsize larger than the first size by stitching the frame images in whichoverlapping portions of the video signal are arranged to overlapaccording to the movement trajectory of the object, and storing thegenerated stitched image and the plurality of frame images as a videofile, and in the storing, the plurality of frame images may be stored tobe played based on the generated stitched image.

According to various embodiments, the generating a stitched image may becharacterized to include identifying a distance according to a movementtrajectory of the object and predicting a size of a stitched image whenframe images are synthesized according to the movement trajectory, andscaling the frame images according to the size of the stitched image,and the scaled frame images are stored.

According to various embodiments, the estimating the movement trajectoryof the object may be characterized to extract the motion vector of theobject motion according to the position of the object for each frameimage, verify whether the motion vectors are valid, and estimate themovement trajectory by connecting validated object motions.

According to various embodiments, the verifying whether the motionvectors are valid may further include calculating the amount of changeof a motion vector corresponding to the motion of the objectcorresponding to the movement of the object, and determining whether theamount of change is equal to or greater than a set threshold to verifyvalidity, for each frame image, and excluding (or dropping) motionvector data of the invalid object motion and generating replacement datato replace the invalid object motion, when it is determined that theobject motion is invalid, as a result of validation verification, andmay be characterized to estimate the movement trajectory of the object,based on the data verified as the valid object motion and thereplacement data.

According to various embodiments, the estimating the movement trajectoryof the object may further include performing trajectory smoothing withthe object motion in which the movement trajectory is smoothed byremoving the small shake data of the movement trajectory.

According to various embodiments, the scaling the frame images may becharacterized to calculate a correction value for scaling the size ofall frame images according to the size of the stitched image, and scalethe size of all frame images by applying the correction value.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic,” “logic block,” “part,” or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, the module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 140) including one or more instructions that arestored in a storage medium (e.g., internal memory 136 or external memory138) that is readable by a machine (e.g., the electronic device 101).For example, a processor (e.g., the processor 120) of the machine (e.g.,the electronic device 101) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a complier or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program product may be traded as a product betweena seller and a buyer. The computer program product may be distributed inthe form of a machine-readable storage medium (e.g., compact disc readonly memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded)online via an application store (e.g., PlayStore™), or between two userdevices (e.g., smart phones) directly. If distributed online, at leastpart of the computer program product may be temporarily generated or atleast temporarily stored in the machine-readable storage medium, such asmemory of the manufacturer's server, a server of the application store,or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities, and some of the multiple entities may beseparately disposed in different components. According to variousembodiments, one or more of the above-described components may beomitted, or one or more other components may be added. Alternatively oradditionally, a plurality of components (e.g., modules or programs) maybe integrated into a single component. In such a case, according tovarious embodiments, the integrated component may still perform one ormore functions of each of the plurality of components in the same orsimilar manner as they are performed by a corresponding one of theplurality of components before the integration. According to variousembodiments, operations performed by the module, the program, or anothercomponent may be carried out sequentially, in parallel, repeatedly, orheuristically, or one or more of the operations may be executed in adifferent order or omitted, or one or more other operations may beadded.

What is claimed is:
 1. An electronic device comprising: a display; amemory; and a processor, wherein the memory comprises instructions forcausing the processor to, when generating a video: receive a videosignal including an subject to obtain a plurality of frame images, eachframe image of the plurality of frame images having a first size;estimate a movement trajectory of the subject included in the pluralityof frame images; stitch together the frame images in which overlappingportions of the video signal are arranged to overlap according to themovement trajectory of the subject to generate a stitched image having asecond size larger than the first size; and store the generated stitchedimage and the plurality of frame images as a video file.
 2. Theelectronic device of claim 1, wherein the memory further comprisesinstructions configured to cause the processor to: store the frameimages to be played, based on the generated stitched image; call thestitched image of the second size to the display when playing the videofile; and display data of the stitched image being generated using pixeldata of the frame images.
 3. The electronic device of claim 1, whereinthe memory further comprises instructions configured to cause theprocessor to: identify a distance according to the movement trajectoryof the subject and predict the second size of the stitched image in casethat frame images are synthesized according to the movement trajectory;and scale the plurality of frame images to fit the size of the stitchedimage.
 4. The electronic device of claim 1, wherein the stitched imageis an image obtained by arranging the plurality of frame images in avirtual two-dimensional or three-dimensional space, and stitchingoverlapping portions of adjacently arranged frame images.
 5. Theelectronic device of claim 4, wherein the stitched image is generated ina size having four vertices where outermost boundary lines of frameimages in which parts having a same image signal among the frame imagesare arranged in an overlapping form in a two-dimensional image areextended and met.
 6. The electronic device of claim 1, wherein thememory further comprises instructions configured to cause the processorto: extract a motion vector of an object motion according to a positionof the subject for each frame image; and verify whether each motionvector is valid to generate a respective validated object motion,wherein the estimate of the movement trajectory is obtained byconnecting the validated object motions.
 7. The electronic device ofclaim 6, wherein the memory further comprises instructions configured tocause the processor to calculate an amount of change of a motion vectorcorresponding to a motion of the subject corresponding to the movementof the subject, and determine whether the amount of change is equal toor greater than a set threshold to verify validity, for each frameimage.
 8. The electronic device of claim 7, wherein the memory furthercomprises instructions configured to cause the processor to: identifyone or more invalid object motions based on a respective motion vectorhaving an associated amount of change that is less than the setthreshold; exclude motion vector data of the invalid object motion andgenerate replacement data to replace the invalid object motion; andestimate the movement trajectory of the subject, based on the dataverified as the valid object motion and the replacement data.
 9. Theelectronic device of claim 6, wherein the memory further comprisesinstructions configured to cause the processor to perform trajectorysmoothing with the object motion in which the movement trajectory issmoothed by removing small shake data of the movement trajectory. 10.The electronic device of claim 3, wherein the memory further comprisesinstructions configured to cause the processor to calculate a correctionvalue for scaling the size of all frame images according to the size ofthe stitched image, and scale the size of all frame images by applyingthe correction value.
 11. A method of generating a video by recording amoving subject using an electronic device, the method comprising:receiving, with the electronic device, a video signal including ansubject to obtain a plurality of frame images, each frame image having afirst size; estimating a movement trajectory of the subject included inthe plurality of frame images; generating a stitched image having asecond size larger than the first size by stitching the frame images inwhich overlapping portions of the video signal are arranged to overlapaccording to the movement trajectory of the subject; and storing, in amemory of the electronic device, the generated stitched image and theplurality of frame images as a video file, wherein, in the storing, theplurality of frame images are stored to be played based on the generatedstitched image.
 12. The method of claim 11, wherein the generating astitched image further comprises: identifying a distance according to amovement trajectory of the subject and predicting a size of the stitchedimage with frame images that are synthesized according to the movementtrajectory; and scaling the frame images according to the size of thestitched image, wherein the scaled frame images are stored in a memoryof the electronic device.
 13. The method of claim 11, furthercomprising: extracting a motion vector of motion of the subjectaccording to a position of the subject for each frame image; verifyingwhether each motion vector is valid; and generating respective validatedobject motion for each frame image, wherein estimating the movementtrajectory comprises connecting validated object motions.
 14. The methodof claim 13, wherein the verifying whether the motion vectors are validcomprises: calculating an amount of change of a motion vectorcorresponding to a motion of the subject corresponding to a movement ofthe subject, and determining whether the amount of change is equal to orgreater than a set threshold to verify validity, for each frame image;identifying one or more invalid object motions based on a respectivemotion vector having an associated amount of change that is less thanthe set threshold; excluding motion vector data of the invalid objectmotion and generating replacement data to replace the invalid objectmotion; and estimating the movement trajectory of the subject, based onthe data verified as the valid object motion and the replacement data.15. The method of claim 11, wherein the estimating the movementtrajectory of the subject comprises performing trajectory smoothing withan object motion in which the movement trajectory is smoothed byremoving small shake data of the movement trajectory.
 16. The method ofclaim 12, wherein the scaling the frame images comprises calculating acorrection value for scaling the first size of all frame imagesaccording to the second size of the stitched image, and scaling thefirst size of all frame images by applying the correction value.
 17. Amethod of generating a video comprising: obtaining a plurality of frameimages with a camera of an electronic device, wherein each frame imagehas a first size; identifying an subject in the plurality of frameimages; estimating a movement trajectory of the subject across theplurality of frame images; generating a stitched image by stitchingtogether at least some of the plurality of frame images based on theestimated movement trajectory, wherein the stitched image has a secondsize that is greater than the first size; storing, in a memory of theelectronic device, the generated stitched image and the plurality offrame images as a video file, wherein the plurality of frame images areconfigured to be played as a video based on the generated stitchedimage.
 18. The method of claim 17, wherein the stitched image isgenerated by stitching together multiple of the plurality of frameimages, wherein frame images having overlapping portions of a videosignal are arranged to overlap according to the movement trajectory ofthe subject.
 19. The method of claim 17, further comprising: extractinga motion vector of motion of the subject according to a position of thesubject for each frame image; verifying whether each motion vector isvalid; and generating respective validated object motion for each frameimage, wherein estimating the movement trajectory comprises connectingvalidated object motions.
 20. The method of claim 19, wherein theverifying whether the motion vectors are valid comprises: calculating anamount of change of a motion vector corresponding to a motion of thesubject corresponding to a movement of the subject, and determiningwhether the amount of change is equal to or greater than a set thresholdto verify validity, for each frame image; identifying one or moreinvalid object motions based on a respective motion vector having anassociated amount of change that is less than the set threshold;excluding motion vector data of the invalid object motion and generatingreplacement data to replace the invalid object motion; and estimatingthe movement trajectory of the subject, based on the data verified asthe valid object motion and the replacement data.